Restricted Least Squares

#econometrics #economics

Oh, Hyunzi. (email: wisdom302@naver.com)
Korea University, Graduate School of Economics.
2024 Spring, instructed by prof. Kim, Dukpa.

Main References

Kim, Dukpa. (2024). "Econometric Analysis" (2024 Spring) ECON 518, Department of Economics, Korea University.
Davidson and MacKinnon. (2021). "Econometric Theory and Methods", Oxford University Press, New York.

Restricted OLS estimates

Proposition (restricted OLS estimates).

Given the prior restrictions as a form of where is matrix and is vector, the restricted least squares estimator is where is the unrestricted OLS and .

Proof.From the given restriction , the restricted OLS is obtained by where the is the Lagrangian constant. Note that the second equation holds since is scalar, and the last equation holds since and is scalar.

F.O.C. where is the restricted OLS estimator.

Since , by multiplying on both sides of , we have since , and by letting , we have Substituting the result back in F.O.C., we have This completes the proof. □

Residual Sum of Squares

Remark (RSS on restricted and unrestricted).

Except if , we have where refers to Residual Sum of Squares, driven as .

Proof.First, remark that the RSS of unrestricted regression is where the third equation holds since is scalar.

Secondly, remark that by ^a8dd9e Proposition 1 (restricted OLS estimates), where . Also, note that is symmetric since Now we drive the restricted RSS. Note that the last equation holds as .
Finally, we show that is positive definite. Since is full rank by A3 A3, we have where the inequality does not holds since is assumed. Therefore, except if . □

Remark (F-test).

Given the restriciton , let Then the F-statistics is given as where .

The proof is given in the section Inferences in Linear Regression > Joint-Hypothesis Test.

Biased Restricted Estimator

Remark (perfect constraints).

If satisfies the constraints exactly, then the restricted OLS is equivalent to the unrestricted OLS. i.e.

Remark (restricted estimator is biased).

Unless , the restricted estimator is biased, i.e.

Proof.From ^a8dd9e Proposition 1 (restricted OLS estimates), we have Therefore, by taking conditional expectation, we have, where the second equality holds by by Finite Sample Results > ^80fc91 Finite Sample Results > Proposition 4 (unbiasedness of least-squares estimate). □

Underestimated Conditional Variance

Remark (conditional variance of restricted and unrestricted).

Whether or not the restrictions are true, we have the following inequality of

Proof.By ^a8dd9e Proposition 1 (restricted OLS estimates), we have Remark that from ^56e25c Remark 5 (restricted estimator is biased), Also, note that where the third equation holds since

Last remark is from Finite Sample Results > ^b90fc8 Finite Sample Results > Lemma 14 (conditional variance of least squares estimate), Finally, we have where the sixth equality holds since Note that is positive semidefinite, since Therefore, we have which completes the proof. □

Underestimated Mean Squared Error

Definition (Mean Squared Error).

The Mean Squared Error (MSE) of the estimate is driven as where is the true parameter.

Remark (MSE on restricted and unrestricted).

Whether or not the restrictions are true, we may have the following inequality of

Proof.Note that MSE of unrestricted OLS is Also, remark that

Then, we have Since and similarly, Furthermore, Therefore, if then holds. □

Conversion into Unrestricted Regression

Suppose the prior restrictions are given as where is and is known constants, for the linear regression Let where is non-singular and the partitions are conformable. Then,

By pre-multiplying , we have which is in vector form, Putting back into the original regression, Therefore, we have where